Co-occurrence of color and luminance edges in natural scenes

Spatial resolution for color is poor, and reports that V1 simple cell respond to oriented achromatic stimuli have led to the view of two independent processing streams, a high-resolution achromatic form system and a low-resolution unoriented color surface system. Recent data have challenged this view, showing that V1 simple cells respond also to oriented chromatic stimuli [5, 7] and that color is processed both in the dorsal and ventral stream [2]. Here we analyze the statistical distribution of luminance and chromatic edges in natural scenes. We show that few edges are characterized exclusively by luminance or chromatic information alone. Instead, the vast majority of edges combine luminance and chromatic information, supporting recent physiological findings of the joint processing of chromatic and luminance information. While few edge are characterized by a pure chromatic contrast, these edge are important for particular ecologically relevant tasks such as the detection of ripe fruit against foliage. Overall, we suggest that chromatic information makes edge detection more robust and provides important additional information for the proper segmentation of objects. The detection of edges is frequently one of the first processing steps both in artificial and natural systems. Traditionally, this process is conceptualized in neurophysiological theories and computationally realized in image processing systems as an achromatic process. input image luminance edges L-M edges Fig. 1: Image of a fruit and the edges detected in the luminance plane and the L-M plane which signals reddish-greenish variations. While the object contour is faintly if at all represented by the luminance edges, a strong response occurs in the L-M plane which almost perfectly delineates the object. Chromatic information helps to separate objects from background. However, important information about object boundaries is sometimes represented only in chromatic channels. Consider the image of a red fruit on green foliage (Fig. 1): In the luminance image, the edges of the fruit are hardly detectable, because the luminance of the fruit is almost the same as the luminance of the background foliage. Any image processing system which tries to detect objects based on luminance information alone would probably miss the fruit. Adding chromatic information changes the situation completely. In the L-M channel, which represents reddish-greenish signal variations, the object boundaries of the fruit is almost perfectly represented. An image processing system which can use this chromatic information will probably detect the fruit. It has been suggested that the ability to process chromatic information and red-green variation in particular has evolved for precisely this task, namely to detect ripe fruit against a background of green foliage [1,14]. More recently, it has been shown that the spectral sensitivities of photo pigments in primates are optimal for the detection of finding fruit or young tasty leaves among a background of mature leaves [16,17]. Here we investigate the more general hypothesis that the advantage of having access to chromatic information is not limited to the detection of fruits and leaves, but instead applies to any objects which have a different color than the background. By combining luminance information with chromatic information the detection of objects becomes easier and more robust. However, this would only make sense if many object borders are defined by a combination of luminance and chromatic contrast. A parsimonious system may still rely on luminance information alone, but taking chromatic information into account would make the detection more robust and reliable. In the following we focus on edge detection, since edge detection is one of the first processing step in object recognition, both in artificial and natural vision system. In particular, we investigate the statistics of luminance and chromatic edges in natural scenes to asses the possible contributions of chromatic information to edge detection. In particular, one wants to know whether all information about edges is essentially contained in the achromatic version of the image, or, alternatively, if and what kind of information can only be obtained form the chromatic channels. The ideas is as follows: A full chromatic images is first separated into a pure luminance image plane and two chromatic images, one varying between reddish-greenish, the other between purple-chartreuse. In this step it is important that the achromatic images are isoluminant, i.e, that they contain no luminance differences. Clearly, a simple difference of RGB channels cannot achieve this property. Instead, one has to use a color space where luminance and chromatic information is properly separated. Here we use the DKL color space [3, 10] whose three cardinal axes (achromatic luminance; reddish-greenish or L-M; and purple-chartreuse or S-(L+M)) are derived from the peak sensitivities of neurons in the LGN, i.e, at the second major processing stage after the initial absorption by the three cone classes. After this transformation, edges are detected in each of the three images planes. Next, the edges within, say, the reddish-greenish image plane are compared to the edges in the achromatic images: for every images pixels we evaluate the strength of the edge in the achromatic image and strength of the corresponding pixel in the reddish-greenish plane. The joint histogram of the edge strength in two images is a formalization of this idea. The joint histograms show that few edges are characterized exclusively by luminance or chromatic information alone. Instead, the vast majority of edges combine luminance and chromatic information. A joint processing of luminance and chromatic information, as supported by recent physiological findings, will thus result in more robust and reliable edge detection and object recognition. Chromatic signals provide a general advantage, which is not limited to special tasks such as the detection of ripe red fruits against green foliage. Chromatic information makes edge detection more robust and provides important additional information for the proper segmentation of objects.